An Audio-Visual Separation Model Integrating Dual-channel Attention Mechanism

Sound source separation is the of targeted sounds from a noisy environment, which plays an important role in signal processing and has been studied extensively. However, most these researches only extract audio information for processing, ignoring visual information, resulting waste feature information. In addition, some researchers have fused extracted features but not noticed weights different features, poor model effect. This paper uses multi-modal to separate sound sources solve problems. We constructed Audio-Visual integrating Dual-channel Attention mechanism named AVDA. The realizes through dynamic fusion features. Specifically, firstly, we take video as input data preprocessing segment it obtain frames audio. Then they are introduced into extractors, integrate attention Finally, prediction component predicted spectrogram, compared with ground truth spectrogram subjective evaluation. three indexes distortion ratio (SDR), interference (SIR) artifact (SAR) quantitative comparison. experimental results on MUSIC-21 set show that achieves 10.96, 17.91 12.77 respectively indexes, its performance significantly better than other models audio-visual tasks.